Skip to content

[DOCS] Document dimension mapping parameter#75414

Merged
jrodewig merged 1 commit intoelastic:masterfrom
jrodewig:docs__add-dimension-mapping-param-docs
Jul 19, 2021
Merged

[DOCS] Document dimension mapping parameter#75414
jrodewig merged 1 commit intoelastic:masterfrom
jrodewig:docs__add-dimension-mapping-param-docs

Conversation

@jrodewig
Copy link
Copy Markdown
Contributor

@jrodewig jrodewig commented Jul 16, 2021

@jrodewig jrodewig changed the title [DOCS] Document time series dimension mapping parameters [DOCS] Document time series dimension mapping parameter Jul 16, 2021
@jrodewig jrodewig changed the title [DOCS] Document time series dimension mapping parameter [DOCS] Document dimension mapping parameter Jul 16, 2021
Changes:
* Documents the `dimension` mapping parameter for `ip`, `keyword`, and `numeric`
  fields.

* Documents the `index.mapping.dimension_fields.limit` index setting.
@jrodewig jrodewig marked this pull request as ready for review July 16, 2021 14:37
@jrodewig jrodewig requested a review from csoulios July 16, 2021 14:37
@jrodewig
Copy link
Copy Markdown
Contributor Author

jrodewig commented Jul 16, 2021

Hi @csoulios.

During a recent meeting, the team agreed to re-add these docs. Let me know what you think. Thanks!

@jrodewig jrodewig added :StorageEngine/TSDB You know, for Metrics >docs General docs changes labels Jul 16, 2021
@elasticmachine elasticmachine added the Team:Docs Meta label for docs team label Jul 16, 2021
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-docs (Team:Docs)

@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jul 16, 2021
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

Copy link
Copy Markdown
Contributor

@csoulios csoulios left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@jrodewig
Copy link
Copy Markdown
Contributor Author

Thanks @csoulios!

@jrodewig jrodewig merged commit 1f04319 into elastic:master Jul 19, 2021
@jrodewig jrodewig deleted the docs__add-dimension-mapping-param-docs branch July 19, 2021 15:24
elasticsearchmachine pushed a commit that referenced this pull request Jul 19, 2021
…5475)

Changes:
* Documents the `dimension` mapping parameter for `ip`, `keyword`, and `numeric`
  fields.

* Documents the `index.mapping.dimension_fields.limit` index setting.
# Conflicts:
#	docs/reference/mapping/types/ip.asciidoc
#	docs/reference/mapping/types/keyword.asciidoc
#	docs/reference/mapping/types/numeric.asciidoc

[[index-mapping-dimension-fields-limit]]
`index.mapping.dimension_fields.limit`::
For internal use by {kib} only. Maximum number of time series dimensions for the
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how it relates to Kibana?

Copy link
Copy Markdown
Contributor Author

@jrodewig jrodewig Jul 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We previously discussed marking the dimension parameter (and by extension this setting) for internal use by the Elastic Observability (specifically metrics) solution only. That solution is shipped as a part of Kibana.

If that's changed, let me know. I can update this in a separate PR. Thanks!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context. My understanding is that this was indeed the initial vision, but that we moved from per-field metadata to dedicated mapping parameters with the vision that these parameters would be used by Elasticsearch in the long term as well. E.g. rollups could auto-configure based on what fields are marked as dimensions, and the timeseries_mode setting could be computed automatically based on dimensions. None of these have been merged yet but hopefully this dimension flag is going to be used by more than Kibana in the near future. @csoulios Please keep me honest if the above is incorrect.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpountz your understanding is correct. dimension and metric mapping parameters will be used by ES (both for TSDB and rollups) for discovering dimensions and metrics. Eventually, they will be available to end-users too.

It's only that at this point we want to keep this feature for internal use only until we feel confident there will be no changes required. See comment: #74450 (review)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the discussion. That aligns with my understanding. The important point is that end users should not be using the dimension parameter at this time.

I'll push a change to broaden the copy to "For internal use by Elastic only."

ywangd pushed a commit to ywangd/elasticsearch that referenced this pull request Jul 30, 2021
Changes:
* Documents the `dimension` mapping parameter for `ip`, `keyword`, and `numeric`
  fields.

* Documents the `index.mapping.dimension_fields.limit` index setting.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>docs General docs changes :StorageEngine/TSDB You know, for Metrics Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Docs Meta label for docs team v7.15.0 v8.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants